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Abstract 


Mi Good translatability of behavioral measures of affect (emo- 
tion) between human and nonhuman animals is core to compar- 
ative studies. The judgment bias (JB) task, which measures 
“optimistic” and “pessimistic” decision-making under ambiguity 
as indicators of positive and negative affective valence, has been 
used in both human and nonhuman animals. However, one key 
disparity between human and nonhuman studies is that the for- 
mer typically use secondary reinforcers (e.g., money) whereas the 
latter typically use primary reinforcers (e.g., food). To address this 
deficiency and shed further light on JB as a measure of affect, we 
developed a novel version of a JB task for humans using primary 
reinforcers. Data on decision-making and reported affective state 
during the JB task were analyzed using computational modeling. 
Overall, participants grasped the task well, and as anticipated, 


INTRODUCTION 


An important goal in cognitive neuroscience, psychophar- 
macology, and affective science is the development of 
translational tasks that can be used to assess affective 
(emotional) states. For example, the development of novel 
treatments for affective disorders in humans depends on 
the use of animal models of affect (Rupniak, 2003). 
Because we cannot simply ask animals to report their af- 
fective state, proxy indicators such as tests of anhedonia 
(Van der Harst & Spruijt, 2007) or cognitive bias (Mend, 
Burman, Parker, & Paul, 2009) are often designed to sim- 
ulate and assess behavioral characteristics observed in 
humans experiencing affective states such as depression 
or anxiety (American Psychiatric Association, 2013; 
Williams, Mathews, & MacLeod, 1996; Wright & Bower, 
1992; MacLeod, Mathews, & Tata, 1986). The judgment 
bias task has been demonstrated to provide a measure of 
affective valence (positivity or negativity) in a range of non- 
human animals with relatively “optimistic” (risk-seeking) 
decision-making being associated with environmental 
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their reported affective valence correlated with trial-by-trial varia- 
tion in offered volume of juice. In addition, previous findings from 
monetary versions of the task were replicated: More positive pre- 
diction errors were associated with more positive affective va- 
lence, a higher lapse rate was associated with lower affective 
arousal, and affective arousal decreased as a function of number 
of trials completed. There was no evidence that more positive va- 
lence was associated with greater “optimism,” but instead, there 
was evidence that affective valence influenced the participants’ 
decision stochasticity, whereas affective arousal tended to influ- 
ence their propensity for errors. This novel version of the JB task 
provides a useful tool for investigation of the links between pri- 
mary reward and punisher experience, affect, and decision- 
making, especially from a comparative perspective. J 


or pharmacological manipulations designed to induce 
positive affect and relatively “pessimistic” (risk-averse) 
decision-making being associated with environmental or 
pharmacological manipulations designed to induce more 
negative affect (Lagisz et al., 2020; Neville, Nakagawa, etal., 
2020). Since its conception in 2004 by Harding, Paul, 
and Mendl (2004), several studies have used judgment 
bias in nonhuman animals as a translational tool to inves- 
tigate human affective disorders and pharmacological 
treatments for such disorders (Hales, Robinson, & 
Houghton, 2016; Anderson, Munafo, & Robinson, 2013; 
Papciak, Popik, Fuchs, & Rygula, 2013; Enkel et al., 2010). 

Conducting judgment bias studies with humans, who 
can report how they feel, may help to provide a better 
insight into how performance in judgment bias tasks 
reflects affective state. It might also help to elucidate the 
putative adaptive function of affect-modulated decision- 
making, specifically, leading to an understanding of 
precisely how and why rewarding experiences might lead 
to positive affect and then to “optimistic” decision-making, 
and vice versa. However, human judgment bias studies to 
date have painted a mixed picture. Whereas some studies 
have found that “pessimism” correlates with subjective 
reports of more negative affect (Positive and Negative 
Affect Schedule [Paul et al., 2010], State-Trait Anxiety 
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Inventory [Aylward, Hales, Robinson, & Robinson, 2020; 
Anderson, Hardcastle, Munafo, & Robinson, 2012], Beck 
Depression Inventory [Daniel-Watanabe, McLaughlin, 
Gormley, & Robinson, in press], Visual Analog Scale for 
anxiety [Anderson et al., 2012]), other studies have found 
no relationship between reported affect and judgment 
bias (affect grid and Positive and Negative Affect Schedule 
[ligaya et al., 2016], State-Trait Anxiety Inventory [Daniel- 
Watanabe et al., in press]) or even that “optimistic” decision- 
making is associated with more negative reported affect 
(affect grid [Neville, Dayan, Gilchrist, Paul, & Mendl, 2021]). 

There are several differences between the human and 
nonhuman animal judgment bias studies that may have 
led to these inconsistencies; humans are required to 
learn the task in a shorter period (i.e., within an hour- 
long session as opposed to over several days for most 
nonhuman animals), humans do not live under the highly 
controlled conditions typical of laboratory animals, and 
there may be social factors influencing human decision- 
making (e.g., wanting to perform well to satisfy the ex- 
perimenter). However, one difference that might be of 
particular significance is that judgment bias studies in 
nonhuman animals are typically conducted using primary 
reinforcers such as food or electric shocks, whereas all 
judgment bias testing in humans has used secondary re- 
inforcers such as monetary gain or loss. 

Many studies have identified differences between the 
neural processing of primary and second reinforcers 
(Sescousse, Caldu, Segura, & Dreher, 2013; Delgado, 
Jou, & Phelps, 2011; Beck, Locke, Savine, Jimura, & 
Braver, 2010; Grimm & See, 2000). For example, it has 
been demonstrated that primary rewards are more 
strongly represented in evolutionary older brain regions, 
such as the anterior insula, whereas secondary rewards 
are more strongly represented in evolutionary newer 
brain regions, such as the anterior OFC (Sescousse 
et al., 2013; Delgado et al., 2011). Given these differ- 
ences, it is important to develop tasks for humans that 
utilize primary reinforcers and hence more closely resem- 
ble animal tasks and to investigate whether they yield 
similar findings to the typical secondary reinforcer hu- 
man task. The use of primary reinforcers in this way is 
particularly pertinent to our understanding of judgment 
bias from a functional and evolutionary perspective, given 
that affect is hypothesized both to reflect ongoing and 
prior experience of rewards and punishers and also to 
mediate the way this experience guides decision-making 
(Bach & Dayan, 2017; Marshall, Trimmer, Houston, 
& McNamara, 2013; Nettle & Bateson, 2012; Mend, 
Burman, & Paul, 2010). Moreover, primary reinforcers 
may tap into fundamental affective processes and pathol- 
ogies that underlie affective disorders more reliably than 
secondary reinforcers, given their more direct relevance 
to our ability to survive and reproduce. Consequently, to 
better understand judgment bias as a measure of affect, 
we need a task for humans in which we administer primary 
reinforcers and in which we can obtain a fine-scale picture 
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of sequential effects that might be influencing decision- 
making and affective state. 

To this end, we aimed to develop a translation of the au- 
tomated rat judgment bias task (Neville, King, et al., 2020; 
Jones et al., 2018) for humans that uses primary rein- 
forcers: apple juice and cold salty tea (Pauli et al., 2015). 
Using this novel variant of the judgment bias task along- 
side computational modeling, we additionally aimed to 
elucidate the extent to which latent processes underlying 
decision-making might relate to subjective experiences of 
affect, reward, and punishment within the task. We hy- 
pothesized that more positive reported affect would be 
associated with model parameters characterizing biases 
toward the “optimistic” response and that, consistent with 
previous research (Neville et al., 2021; Rutledge, Skandali, 
Dayan, & Dolan, 2014), the absolute favorability (i.e., aver- 
age earning rate) of within-task experience would inform 
decision-making and the relative favorability (i.e., reward 
prediction error) of within-task experience would inform 
reported affective valence. 


METHODS 
Participants 


Thirty-three people from the Bristol Veterinary School 
community participated in the study. All participants 
provided written informed consent, and the study was 
approved by the Faculty of Science Research Ethics 
Committee at the University of Bristol. This sample size 
was based on a previous human judgment study (39 par- 
ticipants, across two conditions [Neville et al., 2021]) anda 
previous study using primary reinforcers as part of a con- 
ditioning paradigm (29 participants [Pauli et al., 2015]). 

The inclusion criteria were that the participant enjoyed 
apple juice; was not allergic to apple juice, salt, or black 
tea; was not hypertensive or had any medical condition 
that meant that he or she must limit his or her salt intake; 
and was over the age of 18 years. Participants were asked 
to abstain from drinking anything (except water) or eating 
in the hour before the study. To cover their time and 
expenses, participants were paid &7 for the hour-long 
session. To encourage full engagement with the study, 
participants were informed that the top three ranking 
participants in terms of accuracy would receive an addi- 
tional &7 bonus. 


Apparatus 


The task was conducted ona laptop (Dell Latitude), which 
was connected to two syringe pumps (SPLG100, World 
Precision Instruments) that were set to pump liquid at a 
rate of 2 mL per minute. Sterilized food-grade PVC tubing 
(outer diameter: 11 mm, inner diameter: 8 mm) were 
attached to syringes (Becton Dickinson; 50-mL Plastipak) 
that were driven by the syringe pumps. This tubing was 
connected via a tube connector to smaller sterilized 
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food-grade PVC tubing (outer diameter: 6 mm, inner 
diameter: 4 mm) and held in place in front of the partici- 
pant using a retort clamp and stand, the height of which 
could be adjusted by the participant at the start of the ex- 
periment. The participants placed the end of these tubes 
in their mouth. Individuals made responses on a keyboard 
connected to the laptop. The code for the task was written 
in MATLAB (MathWorks, Inc.) using the PsychToolBox 
extensions (Kleiner et al., 2007; Brainard, 1997). 


Procedure 


The methodology was adapted from that of the human 
monetary judgment bias task (Neville et al., 2021), which 
itself was a translation of a rodent judgment bias task 
(Jones et al., 2018). On each trial of the task, participants 
were instructed to press and hold the enter key before 
being presented with a fixation cross for 1000 msec 
followed by a random dot kinematogram (RDK) dis- 
played for 2000 msec, which across trials varied in direc- 
tion of motion (leftward or rightward) and coherence 
(proportion of dots moving in a coherent direction: 
0.01, 0.02, 0.16, or 0.32). Participants had two options 
when the RDK was presented: to release the key before 
the end of the 2-sec RDK presentation (“leave”) or con- 
tinue to hold the key for 2 sec (“stay”). The outcome as- 
sociated with either response depended on the stage of 
training and the direction of the RDK, with one direction 
being favorable and requiring a “stay” response to gain 
reward and the other being unfavorable and requiring a 
“leave” response to avoid punishment (either leftward or 
rightward, counterbalanced across participants). 

Participants were provided with written instructions 
about the rules of the task and were then provided with 
practice trials that were composed of two blocks of 48 
trials. The aim of the first practice block was to introduce 
the participants to the task and train them on the correct 
responses to the RDK. In this first practice block, the word 
“correct” was shown on screen for correct responses (i.e., 
those where “stay” was executed when the RDK direction 
was favorable, and “leave” was executed when the RDK 
direction was unfavorable), and “incorrect” was shown 
on screen for incorrect responses. The direction of motion 
of the RDK was very easy to detect (coherence = 0.32) on 
all trials, and the direction of motion was leftward on 50% 
of trials and rightward on the remaining 50%. 

The aim of the second practice block was to acquaint 
participants with the delivery and taste of the apple juice 
(Apple Juice from Concentrate, Morrisons) and salty tea. 
Salty tea was prepared each morning as per Pauli et al. 
(2015) with two black tea bags (Bettys & Taylors of 
Harrogate; Yorkshire Tea) and 29 g of salt per liter of 
boiling water, which was chilled before data collection. 
We opted to use salty tea as opposed to electric shocks 
as we considered this to be a milder punisher (hence less 
of an ethical concern) and so that the modality of the 
punisher was the same as the reward (i.e., both gustatory). 


Importantly, liquid reinforcers such as juice and salty tea 
have successfully been used in fMRI studies (Pauli et al., 
2015; Metereau & Dreher, 2013; Kim, Shimojo, & 
O’Doherty, 2011), hence their use in our task paves the 
way for future investigations of the neural underpinnings 
of judgment bias in humans. In this block, the direction of 
the RDK was easy to detect (coherence = 0.16) on 50% of 
trials, of which half were leftward and half were rightward, 
and moderately difficult to detect (coherence = 0.02) on 
the remaining 50%, of which half were leftward and half 
were rightward. The volume of juice received for “stay” 
responses when the direction of the RDK was favorable 
was 0.457 mL, and likewise, the volume of salty tea 
received for “stay” responses when the RDK was unfavor- 
able was 0.457 mL. While juice was delivered and also for 
3000 msec after delivery, the words “Juice delivered” were 
displayed on screen, whereas during salty tea delivery and 
for 3000 msec after this, the words “Salty tea delivered” 
were displayed on screen. The potential volume of juice 
on each trial (i.e., “0.457 mL”) was shown above an 
orange-colored bar with a height proportional to the 
potential juice volume, similarly the volume of salty tea 
(i.e., “0.457 mL”) was displayed below a brown bar posi- 
tioned directly below the orange-colored bar with a height 
proportional to the potential tea volume. These bars were 
shown before the instruction to press and hold the enter 
key. The participant received nothing for making the 
“leave” response, and the words “Nothing delivered” were 
displayed on screen for 3000 msec. Across all blocks, the 
directions and coherence levels of the RDKs were ran- 
domized across trials before the start of the study so that 
the order of these was identical for all participants. 

The test session (see Figure 1) comprised 90 trials on 
which the RDK moved leftward and 90 trials on which it 
moved rightward. For each direction, 30 trials had coher- 
ence levels of low (0.01), moderate (0.02), and high 
(0.16). RDKs with low and moderate coherence levels 
are the ambiguous probe cues, whereas those with a high 
coherence level are the reference cues. Hence, each pos- 
sible stimulus was shown on 16.7% of trials. The potential 
volume of juice fluctuated across trials according to a 
noisy sine wave with a mean volume of 0.457 mL and a 
standard deviation of 0.216 mL, ranging from a minimum 
of 0.021 mL to a maximum of 0.781 mL. As a result of 
ethical concerns about the effect of large quantities of 
salty tea, the potential volume of salty tea remained at 
0.457 mL throughout testing. As in the second practice 
block, the potential volume of juice and salty tea was dis- 
played on screen both as text and graphically using two 
colored bars with heights proportional to the volume of 
juice and salty tea. Before the first trial and every subse- 
quent 10 trials, participants were asked to report how 
they were feeling using a computerized affect grid 
(Killgore, 1998). To complete the affect grid, participants 
had to move a cross that was initially central in the grid to 
the location that best described their current affective 
state using the arrow keys on a keyboard. Horizontal 
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Figure 1. Structure of the human primary reward and punisher judgment bias test session: (1) Participants are shown the potential outcomes of the 
“stay” response (juice volume: orange bar; tea volume: brown bar) and then must press “enter”; (2) participants are instructed to press and hold the 
enter key; (3) participants are shown a fixation cross for 1000 msec; (4) participants are presented with an RDK for 2000 msec, during which they 
must either continue holding the enter key (“stay”) or release the enter key (“go”); (5) participants receive the reward or punishment and are shown 
the outcome of their action (which is also determined by the true direction of the RDK); and (6) after at least 3000 msec, either the next trial starts or 


the participant is asked to complete an affect grid (after every 10 trials). 


movements represented changes in affective valence, 
with movements to the right reporting a more positively 
valenced affect. Vertical movements represented arousal, 
with upward movement reporting higher levels of arousal. 


Data Analysis 


Two participants were excluded from data analysis because 
of poor performance at the reference cues, which we 
defined as not making the correct response significantly 
above chance across both reference cues according to a 
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one-tailed binomial test (Participant 12, p = .18; Participant 
29, p = .99). 

We conducted both a model-dependent and model- 
agnostic analysis of the data. The aim of the model- 
dependent analysis was to investigate the latent processes 
involved in decision-making and how these might be 
modulated by different aspects of reward and punisher 
experience. Judgment bias RT data (“‘stay”: 2 sec; “leave”: 
<2 sec) were fitted to the partially observable Markov 
decision process (POMDP) model described in full by 
Neville et al. (2021). Briefly, we consider that participants 
transition through a 2-D state space in which they 
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accumulate evidence about the true direction of the 
presented stimulus as informed by their observations 
(Dimension 1) across the discretized duration of the trial 
(Dimension 2). The probability on a given trial that an 
individual will opt for the safe “leave” response, and 
the speed with which they do so, will depend on their 
transitions through the state space and the value of occu- 
pying each state. 

In the model, the movement through the state space 
and the values are determined by a number of parame- 
ters including those characterizing the psychometric 
function, namely, o, which reflects the ability of the 
participant to discriminate between the stimuli, and yer 
and amp, Which are lapse rates for the reference and 
ambiguous stimuli, respectively; those characterizing 
the hazard function (probability of “timeout,” i-e., time 
elapsed > 2 sec, given “timeout” has not already occurred) 
to account for the possibility of making the “stay” response 
by default because of inaccurate timekeeping (y and ¢); 
and an inverse temperature parameter (B) to reflect deci- 
sion stochasticity. In addition, we included a set of param- 
eters that allowed for biases toward or away from the 
“optimistic” response. This set of parameters was com- 
posed of a baseline bias parameter (>) capturing overall 
tendencies for risk-seeking or risk aversion, as well as the 
overall dislike for the salty tea and enjoyment of the apple 
juice, and parameters that encompassed the potential 
effect of the average earning rate (B°), most recent 
outcome (8%), weighted prediction error (R>»,), and 
squared weighted prediction error (> pp) on bias, as well 
as a parameter that allowed for decision-making to vary as 
a function of the number of trials completed (B,,) to 
account for fatigue. The average earning rate reflects the 
learnt value of the test session from previous juice and 
salty tea intake and updates according to a Rescorla— 
Wagner learning model, with learning rate ag (following 
Neville et al., 2021; Guitart-Masip, Beierholm, Dolan, 
Duzel, & Dayan, 2011), whereas the weighted reward 
prediction error is the difference between the model- 
predicted outcome and the actual outcome across trials 
weighted such that the influence of past prediction errors 
attenuates over trials, with forgetting factor y,pp (following 
Neville et al., 2021; Rutledge et al., 2014). 

As the task used food rewards, we included a parameter 
(k) that captured the potential effect of satiation as juice 
intake increased. Specifically, the subjective worth (R,,) 
depended on the offered juice volume (in milliliters) on 
trial 7 via an exponent that varied as a function of total 


even -1 
juice intake, 0") Ry: 


n-1 


1-k ) R; 


-=0 
R, = Ry ; 

An exponent was chosen to reflect the nonlinearity of util- 
ity functions and, more specifically, to capture the typical 
concavity of utility functions; increases in reward gain lead 


to diminishing marginal increases in the subjective value 
of those gains (Hsee & Rottenstreich, 2004; Kahneman 
& Tversky, 1979). As the offered volume of juice is always 
lower than 1, a negative value of k would represent 
adherence with the law of diminishing marginal utility, 
whereas a value of greater than zero would lead to the 
opposite (i.e., a steep increase in the subjective value of 
the juice with additional consumption). 

Models were fitted to the RT data using maximum like- 
lihood, with multiple starting values (including values 
found to provide the best fit for the core model as starting 
values) because of nonconvexity. Parameters that charac- 
terized decision-making in the absence of biases (i.e., B, 
XNamb; Aref 0, Y, and $) were included in all models to ac- 
count for their potential influence on behavior. We then 
added parameters in a stepwise manner, first assessing 
whether the parameter that characterized constant biases 
in decision-making improved model parsimony and then 
those that characterized within-task variation in decision- 
making. We then assessed whether the addition of single 
parameters to the best-fitting model would further improve 
the model fit. Models were compared using Bayesian infor- 
mation criterion (BIC) values, and we additionally com- 
pared the final set of models using the Akaike information 
criterion (AIC). A stepwise model-fitting procedure was 
employed as fitting all possible models was not feasible 
because of the large number of possible models and the 
computational intensiveness of model fitting. Model fitting 
was carried out using the computational facilities of the 
Advanced Computing Research Centre at the University 
of Bristol. 

The parameter estimates from the most parsimonious 
model were analyzed using permutation tests to assess 
whether they varied significantly from zero, where this 
was meaningful (i.e., for B} and k). 

The aim of the model-agnostic analysis was to investigate 
the relationship, first, between primary reward and 
punisher experience and reported affect and, second, 
between the parameters characterizing decision-making 
and reported affect. These analyses were conducted in R 
(R Core Team, 2015) using the z/me package (Pinheiro, 
Bates, DebRoy, Sarkar, & R Core Team, 2017). Likelihood 
ratio tests were used to assess whether the difference in 
model deviance was significant after removal of a parame- 
ter from a model. 

The reported valence (affect grid x coordinate) and re- 
ported arousal (affect grid y coordinate) were analyzed 
using general linear mixed models (GLMMs) in which 
both the intercept and slope were allowed to vary among 
participants, with the predictor variables for which a ran- 
dom slope was included determined using BIC compari- 
son (i.e., comparing a model with the variable included in 
the random effects structure to the model without the 
variable included in the random effects structure). The 
predictor variables were those encompassing reward 
and punisher experience, included those derived from 
the best-fitting POMDP model: the most recent volume 
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of offered juice, weighted reward prediction error (wPE), 
squared weighted reward prediction error (wPE’), the 
previous outcome O, total juice consumed, the number 
of trials completed, and the average earning rate R. 
Because of correlations between wPE, wPE’, and O, and 
also between total juice consumed, number of trials com- 
pleted, and R, these predictor variables were not included 
in the same GLMM but instead were included separately, 
and each model was then compared using their BIC 
values. The GLMMs that provided the best fit were 
analyzed further. 

In addition, the parameter estimates from the POMDP 
model were analyzed using a general linear model with 
the mean reported arousal and mean reported valence 
as fixed effects, except § and y, in which the mean timeout 
probability that was jointly determined by these two pa- 
rameters was instead used for a more intuitive interpre- 
tation of the results, as in Neville et al. (2021). The values 
of B were log-transformed because of the presence of ex- 
treme outliers exerting undue influence on the model; 
the GLMM was also run after removal of these outliers. 


RESULTS 
Judgment Bias 


The most parsimonious model of judgment bias RT, ac- 
cording to the BIC values, included the following core pa- 
rameters: B, Namb; Mref 0, Y, and ¢, which characterized 
decision-making in the absence of biases, in addition to 
B3, characterizing baseline biases, and «, characterizing 
the rate of satiation (see Table 1). This model fit the ob- 
served data well (see Figure 2). The AIC values indicated 
that inclusion of B8 or 8°», outperformed the BIC best 
model. However, permutation tests revealed that the es- 
timates of these parameters did not differ significantly 
from zero (B2: mean = —0.13, SE = 0.11, p = .28; 
Be pp: mean = 0.005, SE = 0.01, p = .72). Hence, there 
was no strong evidence for the inclusion of these param- 
eters, and we selected the BIC best model as our final 
model for further analysis. 


Decision-Making and Reported Affect 


Visual inspection of the parameter estimates revealed 
several potential outliers. These were formally defined 
as estimates that were either above the third quartile or 
below the first quartile by more than 1.5 times the inter- 
quartile range (i.e., below Q1-1.5 X IQR or above Q3 + 
1.5 X IQR). All analyses were conducted both before and 
after the removal of these outliers. The presented results 
are from the analyses including the outliers, except 
where the removal of outliers changed the results quali- 
tatively, in which case both results are presented. 

The parameter characterizing satiation was significantly 
lower than zero (k: mean = —5.84, SE = 4.23, p < .001), 
reflecting the fact that participants valued juice less with 
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Table 1. Comparison of POMDP Models 





Model AIC BIC 

3, Kamp; Aref $, Y, B 15024.49 — 15656.70 
@) hans Newt Ce Os B, BG 13652.29 — 14389.86 
3; Kamps Mes $ 9; B, BG, BR 13600.18  14443.12 
3; Namb: Mes $ 9 B, BG, Bes OR 13560.34 — 14508.65 
Oi, haus Mees © VB, BG, Bie 13646.15  14489.09 
G, Nambs Aref $, 9, B, BGs Bape, ‘Ywrr  13704.29 —- 14652.60 
G, Nam» Mret $Y, B, BB, Beppe 13662.85  14505.80 
O, Namb: Nret § PB, BG, Beppe, Ywre  13590.04 — 14538.35 
;. Nema; Nee $5) Bs BGs BO 13676.61 — 14519.55 
GO, Namb, Nees $, 9, B, BG: Br 13528.49 — 14371.43 
o, Namb, ret $, 9, B, BE, K 12641.70 13484.64 
@, Kamp: Mees $, %, B, Bos *) Bp 12639.33 — 13587.64 
Gs Keaaithi Rnek $0 By BG Rs Boe 12600.02  13548.33 
©, amb; Aref $, ¥, B, BG, K, Bonne 12991.77.  13940.08 
0, Namb, ress & Y, B, BG. k, BY 12700.81 13649.12 
0, Namb; Ares S, 9, B, BR, K, Br 12700.61  — 13648.92 
Values in bold highlight the minimum value for each comparison 
method. 





increasing juice consumption. Overall, the subjective 
value of the juice depreciated across the test session and, 
for much of the test session, remained less appetitive 
than the salty tea was aversive (Figure 3). This decrease 
in the subjective value of the juice exerted a strong influ- 
ence on behavior; participants were more risk-averse 
than would be expected if there was no effect of satiation 
(Figure 4). The effect of satiation was also reflected in 
the observed decrease in RTs and proportion of “stay” 
responses made as the number of trials increased 
(Figure 5). The estimates of k were not associated with 
either reported valence (beta weight = —5.72, SE = 
4.64, likelihood ratio test [LRT] = 2.03, ) = .16) or arousal 
(beta weight = 0.98, SE = 4.64, LRT = 0.075, p = .79). 

The bias parameter was significantly greater than zero 
(B}:; mean = 0.05, SE < 0.01, p < .001), reflecting the par- 
ticipants’ overall bias toward persisting with the “stay” 
response for a longer duration on a trial than would 
otherwise be anticipated (Figure 4), but this parameter 
was not predicted by reported affective valence (beta 
weight < 0.01, SE < 0.01, LRT < 0.01, p = .95) or arousal 
(beta weight < 0.01, SE < 0.01, LRT = 0.55, p = .46). Thus, 
although the value of the juice depreciated considerably 
over trials (Figure 3), participants occasionally continued 
to risk making the “stay” response, which was driven by 
this fixed bias. There was no evidence that the extent of 
this bias, or the speed of satiation, was influenced by 
affective state. 
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Figure 2. Observed versus generated RT data. The mean discretized RT and mean proportion of stay responses for each stimulus level for both the 
model-generated and observed judgment bias data. Error bars represent 1 SE. 


Participants who reported lower arousal tended to 
make more errors to the ambiguous stimulus; the associ- 
ation between the lapse rate and reported arousal was 
marginally nonsignificant (Agmp: beta weight = —0.02, 
SE = 0.01, LRT = 3.77, p = .052). The reference lapse rate 
did not depend on reported arousal (A;¢¢: beta weight < 
0.01, SE < 0.01, LRT = 2.04, p = .15). Neither Namp (beta 
weight < 0.01, SE = 0.01, LRT < 0.01, p = .93) nor Myer 
(beta weight < 0.01, SE < 0.01, LRT = 2.13, p = .15) 


depended on reported valence. The slope parameter, o, 
did not significantly depend on reported affective valence 
(beta weight = 0.04, SE = 0.04, LRT = 1.10, p = .30) or 
arousal (beta weight = 0.03, SE = 0.04, LRT = 0.78, p = 
.38). Likewise, the mean timeout probability (determined 
by ¢ and y) was not significantly associated with reported 
affective valence (beta weight < 0.01, SE = 0.03, LRT < 
0.01, p = .95) or arousal (beta weight = 0.01, SE = 0.03, 
LRT = 0.41, p = .52). 
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Figure 3. The value of juice across trials; the black line reflects the offered value, and the gray line reflects the mean satiation-dependent value across 
participants as determined by the POMDP model. The dashed line shows the threatened volume of salty tea. The shaded error bars represent 1 SE. 
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Participants who reported more positive valence had 
significantly higher inverse temperature parameter values 
(B: beta weight = 1.05, SE = 0.48, LRT = 4.92, p = .027; 
Figure 6), and there was no significant relationship 
between affective arousal and B (beta weight = 0.22, 
SE = 0.48, LRT = 0.23, p = .63). However, after outlier 
removal, no significant effect of valence (beta weight = 
—0.21, SE = 0.30, LRT = 0.52, p = .47) or arousal (beta 
weight = —0.46, SE = 0.28, LRT = 2.97, p = .085) on 
inverse temperature was observed (Figure 6). The median 
value of the inverse temperature parameters was 336.4297 
with an interquartile range of 1224.997, and any value over 
3172.95 was deemed to be an outlier. The six outliers iden- 
tified, whose removal altered the results qualitatively, had 


values of 4099.67, 5309.31, 82248.43, 15654.76, 121346.67, 
and 2779806.37 (Figure 6). 


Reported Affect and Reward and 
Punisher Experience 


The best-fitting GLMM of reported valence included the 
potential outcome as solely a fixed effect (ABIC = 12.27 
vs. inclusion as a random linear effect), number of trials 
completed with a slope coefficient that varied among 
participants (ABIC = 26.91 vs. solely a fixed effect; as 
opposed to the average earning rate — ABIC = 26.75, 
and total juice consumed — ABIC = 1.10), and weighted 
prediction error as solely a fixed effect (ABIC = 14.98 vs. 





Figure 5. Observed versus 
generated RT and decision data. Trial End 
The mean discretized RT 

and mean proportion of 

stay responses for each 
stimulus level for both the 
model-generated and observed 
judgment bias data split by trials 
in the first and second half of 
the test session. Error bars 
represent 1 SE. 
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Figure 6. The relationship 





between the log-transformed 15 
inverse temperature parameter 
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affective valence with regression 
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inclusion as a random linear effect; as opposed to the 
squared weighted prediction error — ABIC = 54.25, or 
previous outcome ABIC = 38.14). Reported valence 
was positively correlated with both the prediction error 
(beta weight = 40.70, SE = 5.34, LRT = 52.14, p < 
.001) and potential outcome (beta weight = 14.30, 
SE = 4.24, LRT = 11.33, p < .001). The random slope 
coefficients for trial did not differ significantly from zero 
according to a permutation test (mean = —0.03, SE = 
0.12, p = 81). 

The best-fitting GLMM of reported arousal included 
potential outcome as solely a fixed effect (ABIC = 
15.34 vs. inclusion as a random linear effect), number 
of trials completed with a slope coefficient that varied 
among participants (ABIC = 71.70 vs. solely a fixed 
effect; as opposed to the average earning rate — ABIC = 
83.80, or total juice consumed — ABIC = 0.95), and 
squared prediction error as solely a fixed effect (ABIC = 
18.37 vs. inclusion as a random linear effect; as opposed 
to the weighted prediction error — ABIC = 1.11, or pre- 
vious outcome — ABIC = 1.12). A higher potential out- 
come tended to be associated with greater reported 
arousal (beta weight = 7.06, SE = 4.66, LRT = 2.89, 
p = .09). The squared weight prediction error had no 
significant effect of affective arousal (beta weight = 
4.96, SE = 4.66, LRT = 1.14, p = .26). The random slope 
coefficients for trial were overall significantly lower than 
zero (mean = —0.52, SE = 0.17, p = .004), suggesting a 
decrease in reported arousal as the test session progressed. 


DISCUSSION 


In this study, we developed a human judgment bias task 
that closely mirrored nonhuman animal versions of the 
task by using primary, rather than secondary, reinforcers. 
We did this to increase the translatability of the task and 


to allow more effective use of results from the human 
task to shed light on judgment bias as a measure of affect 
in nonhuman animals. To achieve this, we modified a hu- 
man judgment bias task, which was itself originally de- 
signed as a translation of a rodent judgment bias task, 
by replacing monetary gain with apple juice and mone- 
tary loss with salty tea. We then sought to investigate 
the relationship between prior experience, affect, and 
decision-making using computational modeling. The task 
itself was largely successful, with more than 90% of 
participants performing better than chance in their 
responses to the reference cues to which they had been 
trained. Furthermore, the data produced were highly 
amenable to computational modeling. 

Our initial analyses demonstrated that the favorability 
of the potential outcome of participants’ responses mod- 
ulated self-reported affective valence, with a greater vol- 
ume of offered juice leading to more positive affective 
valence. There was also a tendency for a greater volume 
of offered juice to lead to greater reported affective 
arousal. This supports the definition of “emotion” (i.e., 
short-term affect) as a state elicited by (anticipated) 
rewards or punishers (Mendl & Paul, 2020; Rolls, 2013). 
It also indicates that participants were sensitive to the 
fluctuating volume of potential apple juice and were, 
indeed, perceiving larger volumes of apple juice as more 
rewarding. This demonstrates that the manipulation of 
reward experience worked well, having induced the 
anticipated shifts in affective states. The finding that 
positive affective valence was related to recently experienc- 
ing rewards that were greater than anticipated (positive 
reward prediction errors) aligns with the results of a num- 
ber of other studies (Neville et al., 2021; Otto & Eichstaedt, 
2018; Rutledge et al., 2014). 

Our results provide evidence for a complex relation- 
ship between affect and decision-making that could only 
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be teased apart through use of computational modeling. 
Contrary to our expectations, there was no evidence in 
this study that an “optimistic” bias was directly associated 
with more positive reported affect. This conflicts with 
some (but not all) previous findings from judgment bias 
studies in humans and with the meta-analytic findings in 
animal studies that relatively positive affective states are 
associated with more “optimistic” decision-making 
(Lagisz et al., 2020; Neville, Nakagawa, et al., 2020). 
Importantly, the meta-analyses identified a small-to- 
moderate effect size with high heterogeneity, using 
manipulations including those designed to induce 
substantial shifts in affective valence (e.g., enriched vs. 
barren housing; administration of pharmacological 
substances; restraint stress). It is also possible that the 
primary reinforcers used in these animal studies may 
have been far more salient to their nonhuman subjects 
than the juice/salty tea was for the humans in this study 
(e.g., animals were food restricted; animals were 
maintained on a nonvaried diet). Given this, it is perhaps 
not entirely unsurprising that no significant relationship was 
found between the parameter estimates encompassing 
“optimism” and affective valence in our nonclinical popula- 
tion of human participants. These findings, first, raise the 
question of whether judgment bias can be sufficiently 
sensitive to detect more subtle variation in baseline affect 
and, second, highlights that contextual differences in how 
humans and animals experience the tasks may present 
significant hurdles to the development of truly translational 
and directly comparable tests, even given the explicit 
attempt. 

We did find that, when looking at the data as a whole, 
the inverse temperature parameter was associated with 
reported valence; lower decision stochasticity was associ- 
ated with more positive reported affect. This result is 
consistent with findings that depression, a clinically neg- 
atively valenced affective state, has been associated with 
more stochastic decision-making (Harlé, Guo, Zhang, 
Paulus, & Yu, 2017; Huys, Pizzagalli, Bogdan, & Dayan, 
2013). However, this result appeared to be driven by a 
subset of participants whose data revealed large outlying 
estimates of the inverse temperature parameter and who 
tended to report affective valence at the more positive 
end of the scale. The cause of the outlying values is 
unclear; speculatively, it could reflect a trait characterized 
by heightened sensitivity to rewards and punishers and 
overall more positive affect. It could also reflect that 
the model fits particularly well to these participants 
(given that the inverse temperature parameter can also 
capture model misfit), although the reasons for this are 
not apparent. Nonetheless, the results of our study imply 
a relationship between affective valence and the cogni- 
tive processes underlying judgment bias. The potential 
link between decision stochasticity and affect requires 
further investigation. 

We also identified a weak relationship between affec- 
tive arousal and the cognitive processes underlying 


2532 ‘Journal of Cognitive Neuroscience 


judgment bias. In particular, a higher ambiguous stimulus 
lapse rate tended to be associated with lower arousal, 
which corroborates results from the monetary version 
of the task (Neville et al., 2021) and likely reflects that 
higher arousal may lead to higher engagement with the 
task and result in better performance. It would therefore 
be useful to assess whether lapse rates, extracted using 
computational modeling, could provide a measure of 
arousal in judgment bias tasks for nonhuman animals. 

Our results also highlighted the influence of reward 
and punisher experience on both affect and decision- 
making. First, the total juice consumed by the participant 
was the largest experiential contributor to their decision- 
making. The value of the juice decreased as a function of 
increasing juice intake, which is consistent with partici- 
pants becoming satiated. In accordance with the findings 
of the monetary version of the task (Neville et al., 2021), 
we also found decreasing arousal as more trials had been 
completed, which may indicate that participants found 
the task to be tiring. Greater consideration should thus 
be given to the potential for satiation and potential fa- 
tigue (and individual differences that lead to variation 
in how rapidly an individual becomes sated) when con- 
ducting nonhuman animal judgment bias tasks. 

The degree of satiation indicated in our human partic- 
ipants was arguably extreme; the absolute subjective value 
(i.e., subjective magnitude) of the juice remained largely 
below the value of the salty tea after approximately 20 
trials. Yet, despite this, and even after accounting for errors 
that were characterized by a lapse rate, participants con- 
tinued to make the “optimistic” response more often than 
would be anticipated; they exhibited an overall “optimis- 
tic” bias. The precise nature of this bias is unclear. One pos- 
sibility is that this bias, particularly at the favorable and 
near-favorable cue, reflects the intrinsic reward associated 
with making accurate (correct) responses. Similarly, it may 
reflect that the feedback provided by the “stay” response 
was itself rewarding, given that the “leave” response pro- 
vides no feedback about the correct action. This would 
be in line with studies that have shown that being correct 
is in itself rewarding (Satterthwaite et al., 2012; Han, 
Huettel, Raposo, Adcock, & Dobbins, 2010). An alternative 
is that the “gamble” of the “stay” may have elicited a reward- 
ing frisson of excitement, particularly given the repetitive 
and likely dull nature of the task. Studies have shown that 
risky decision-making can induce excitement and that this 
increases under boredom (Kulic, van Tilburg, & Igou, 2020; 
Binde, 2013; Mercer & Eastwood, 2010). This further high- 
lights issues in developing truly translational versions of the 
judgment bias task; nonhuman animals may not experi- 
ence an intrinsic reward associated with being correct, 
and judgment bias tasks for nonhuman animals have been 
suggested to be enriching for them (Krakenberg et al., 
2021; Mallien et al., 2016). 

The extent to which recent outcomes were better or 
worse than anticipated was found to determine self-reported 
affect. This is in agreement with previous studies 


Volume 33, Number 12 


Zz0z unr €0 uO ysan6 Aq jpd'g//10 & UdOl/EggO/6L/EZSZ/ZL/EeAPd-ejoIe/UOO[/npa yw 'ysuIp//:dyY Woy papeojumoq 


demonstrating that prediction error is a key determinant 
of subjectively experienced affective valence in humans 
(Neville et al., 2021; Otto & Eichstaedt, 2018; Rutledge 
et al., 2014), at least in relation to short-term rewarding 
experiences. 

As outlined above, several findings were consistent 
between this task and the monetary version from which 
it was adapted: A higher lapse rate is associated with 
lower affective arousal, more positive prediction errors 
are associated with more positive reported affective 
valence, and reported affective arousal decreases as a func- 
tion of number of trials completed. However, there were 
also a couple of inconsistencies. First, we found no 
evidence for a relationship between time estimation and 
affective valence, and second, we found no effect of the 
average earning rate or previous outcome on judgment 
bias (Neville et al., 2021). It is unclear whether this might 
stem from differences in the processing of primary and 
secondary rewards, or other factors such as satiation 
having an overriding effect on behavior. This therefore 
warrants further examination. 

In summary, we have developed a novel version of the 
judgment bias task for humans that uses primary as 
opposed to secondary reinforcers and, as a result, is 
more comparable to versions of the task designed for 
nonhuman animals. We identified a relationship between 
decision stochasticity and affective valence. Specifically, 
we generally observed higher decision stochasticity in 
participants who reported more negatively valenced 
affect, although only when a cluster of outlying partici- 
pants were included. We also found that lower reported 
affective arousal tended to be associated with a greater 
propensity for errors. The decision-making processes 
underlying judgment bias on this task are hence linked 
to both reported affective valence and arousal. In addi- 
tion, we confirmed some, but not all, previous findings 
that had been observed in a monetary version of the task, 
such as the positive association between prediction 
errors and reported valence. We conclude that our novel 
version of the judgment bias task provides a means to 
investigate the relationship between affect and decision- 
making in greater depth and in a manner that is more 
comparable to animal versions of the task. 
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